Broadened-Specular Reflection and Linear Subspaces 

for Object Recognition 



Background of the Invention 
Field of Invention 

[0001] The present invention relates generally to machine 
vision, and more particularly, to a method of modeling the 
broadened-specular reflection of light to allow a machine 
to recognize the image of an object illuminated by varying 
lighting conditions . 

Description of Related Art 

[0002] It is clear that changes in lighting can have a 
large effect on how an object, such as a person, looks. 
Understanding this is essential to building effective 
object recognition systems. 

[0003] U.S. Patent Application Serial Number 09/705,507 
(Attorney Docket No. NECI 1093, 13919), filed 3 November 
2000, entitled "Lambertian Reflectance and Linear 
Subspaces", the disclosure of which is hereby incorporated 
by reference, considered the relationship between the 



function that describes the lighting intensity, and the 
reflectance function that describes how much light an 
object reflects as a function of its surface normal, under 
a given lighting condition. Representing these functions 
as spherical harmonics, it was shown that for Lambertian 
reflectance the mapping from lighting to reflectance is a 
convolution with a nearly perfect low-pass filter. High- 
frequency components of the lighting hardly affect the 
reflectance function. Therefore, nearly all Lambertian 
reflectance functions could be modeled as some linear 
combination of nine spherical harmonic components. 

[0004] These low-dimensional approximations have two key 
advantages. First, they are described with few parameters, 
thus optimization is simpler. Moreover, the images 
produced under low-frequency lighting are inherently immune 
to small changes in orientation. A small rotation in the 
surface normal is equivalent to a small rotation in the 
light. However, low-frequency light does not appreciably 
change with a small rotation. The analysis shows this, in 
that an image produced by an n th order harmonic light 
component is computed by taking an 21 th degree polynomial of 
the components of the surface normal. 



[0005] While promising, these results considered only 
Lambertian reflectance. Specular reflectance can 
significantly affect how an object appears. For example, a 
human face may have some oiliness, particularly on the 
forehead, which reflects light specularly. This is seen as 
highlights on the object. 

[0006] Specular reflectance cannot be approximated well by 
only considering low-dimensional linear subspaces which 
capture only low-frequency lighting. Intuitively, the set 
of possible reflectance functions for a perfect mirror is 
the same as the set of possible lighting functions, as will 
be shown, infra. The question remains how to cope with 
specularity in object recognition using models that will 
inevitably be imperfect. 

[0007] Specularities are often thought of as mainly present 
in spatially localized highlights, which occur when a 
bright light source is reflected by a shiny object. Such 
highlights can potentially be identified in the image and 
processed separately. For example, the algorithms 
described in Coleman, et al., Obtaining 3-Dimensional Shape 
of Textured and Specular Surfaces Using Four-Source 
Photometry , Physics-Based Vision Principles and Practice, 
Shape Recovery, pp. 180-199 (1992), and Brelstaff, et al., 



Detecting Specular Reflection Using Lambertian Constraints , 
International Conference on Computer Vision, pp. 297-302 
(1988), treat specular pixels as statistical outliers that 
should be discarded. However, specularities due to low- 
frequency lighting can be spread out throughout the entire 
image. Since these specularities affect every pixel in the 
image, their effect must be better understood; they cannot 
simply be discarded. 

[0008] Some previous works have considered specular effects 
also as convolutions. Basri, et al., (August 2000) 
considered a version of the Phong model in which the 
specular lobe is a rotationally symmetric shape centered 
about a peak that is a 180-degree rotation of the direction 
of .the incoming light about the surface normal. In such a 
model, specular reflection doubles the frequency of the 
incoming light, and then acts as a convolution. However, 
this model is heuristic, and not physically based. It does 
not extend to realistic models. 

[0009] Ramamoorthi, et al., On the Relationship between 
Radiance and Irradiance: Determining the Illumination From 
Images of a Convex Lambertian Object , to appear in Journal 
of the Optical Society of America A, considered the effects 
of arbitrary bi-directional reflectance distribution 
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functions (BRDFs) on the light field. They were able to 
develop algorithms for recovering the BRDF and/or the 
lighting given a measured light field and an object with a 
known 3D structure. Though interesting, these results are 
not generally applicable to machine vision because, without 
special sensing devices, the light field is not available. 
An image of an object is produced by the reflectance 
function that describes how the surface normals reflect 
light in one fixed direction. Therefore, an image 
corresponds to a 2D slice through the light field. For an 
arbitrary BRDF, the 2D reflectance function for a fixed 
viewpoint no longer results from a convolution of the 
lighting. 

Brief Summary of the Invention 

[0010] The goal of the present invention is to allow a 
machine, such as a computer, to match an input image of an 
object, such as a human face, to an object model stored in 
a database associated with the machine, and to perform this 
matching with a high degree of accuracy. 

[0011] The steps of the recognition method of the present 
invention include providing a database of three-dimensional 
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object models, providing an input image, positioning each 
three dimensional model relative to the input image, 
determining an optimal rendered image of the model that is 
most similar to the input image, computing a measure of 
similarity between the input image and the optimal rendered 
image of each model, and selecting the object whose optimal 
rendered image is most similar to the input image. The 
step of determining an optimal rendered image includes 
deriving a reflectance function for each object that 
includes the effects of both broadened-specular and diffuse 
reflection, and optimizing that reflectance function to 
most closely match the input image. 

[0012] The method of the present invention includes 
rendering the images produced by an object model when 
illuminated only by each of a plurality of spherical 
harmonic components of light. The summation of these 
components defines the reflectance function for the object 
model. The derived reflectance function takes into account 
the intensity of light components incident on the object 
model, the intensity of light diffusely reflected from the 
model, and the intensity of light specularly reflected from 
the model, the last component being dependent on the angle 



between the direction of the observer and the direction of 
specular reflection. 

[0013] To overcome the limitations of dealing with specular 
reflection, a two-pronged strategy is used. The light in 
low-frequency and higher-frequency components are analyzed 
separately. The low-frequency components are analyzed 
using the method of the present invention, and the results 
of this analysis are evaluated to determine if they are 
sufficient to match an optimal rendered image of the object 
model to the input image. 

Detailed Description of the Invention 

[0014] Although detailed and/or microscopic models of 
reflection exist, these apply principally to homogeneous, 
rigid, material surfaces. For reflection from 
inhomogeneous, flexible surfaces such as faces, where most 
of 'the features of interest are conf igurational, a more 
robust model that captures the most important aspects of 
the scattered light is more appropriate. Thus the 
Lambertian model, which simply scatters incident radiation 
equally into the hemisphere above each surface element, is 
usually chosen . 



[0015] However, due to broadened-specular reflection, in 
viewing a person's face, regions of unusually high 
reflectivity are apparent. The intensity of light reflect 
from these regions is sensitive to the viewing direction. 
As this aspect of the illuminated face cannot be captured 
by Lambertian scattering, even in principle, the Lambertian 
model is supplemented with the Phong model. The latter 
scatters the incoming light preferentially into directions 
distributed about the direction of purely specular 
reflection. In addition, the distribution of scattered 
directions is readily analyzable in the spherical-harmonic 
representation, in a manner similar to that of the 
Lambertian model. Appropriately weighted, this additional 
and more selective scattering should adequately model the 
human face for most lighting conditions of interest. 

[0016] Though an enhancement of the Phong model of 
reflection was chosen, the method of the present invention 
need not be based upon the Phong model. Any model that 
depends only on the cosine of the angle between the 
reflected light and the direction of perfect specular 
reflection can be treated in the same manner. 

[0017] While not completely dominant, low-frequency 
lighting will have a considerable effect on the 



reflectances due to the broadened-specular component of an 
object* This is because many situations contain 
significant low-frequency lighting, such as outdoors on a 
cloudy day, and many indoor settings. 

[0018] The method of the present invention shows how to 
analyze the reflectance function that is produced by 
specular reflection for more physically realistic BRDFs ♦ 
This is accomplished by adding more realistic terms to the 
Phong model. These terms modulate the Phong model by the 
amount of light that is actually striking a surface patch. 
The new model allows the development of techniques that 
will be relevant for more physically realistic models, 
while avoiding some of the complexity of those models. 

[0019] The physically realistic model leads to the problem 
of .taking integrals in three terms: the lighting intensity, 
which is a function of its incoming direction; the specular 
term, which is a function of the angle between the specular 
peak and the viewing direction; and the amount of light 
striking the surface, which is a function of the angle 
between the incoming light and the surface normal. After 
expanding these terms in spherical harmonics, this can be 
rewritten as the integral of the product of three 
harmonics. This problem has been solved in other areas of 



research, for example atomic spectroscopy. The solution 
may be thought of as a generalization of the convolution 
theorem to this case. 

[0020] The results can be used to determine the set of 
images that an object produces when lighting consists of a 
single harmonic component. Therefore, one can analytically 
characterize the images an object produces, and determine 
how the effect of low- and high-frequency lighting on an 
image varies with the parameters of the Phong model. 

[0021] Conceptually, the method of deriving the reflectance 
function of an object model according present invention 
involves transforming the function of light intensity 
incident upon an object into its low-dimensional spherical 
harmonic components. In the present sense, low-dimensional 
is used to mean the 9D subspace formed by the summation of 
the 0 th , 1 st , and 2 nd order spherical harmonic components. 
Each component is applied to the surface normal at each 
point on the object. Using the reflectance model described 
herein, the reflectance function is determined, and 
described in terms of the same spherical harmonic 
components. These harmonic components can then be summed 
to produce a rendered image, a linear representation of the 
image of the object model. As a practical matter, however, 
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the machine can utilize the harmonic components and 
optimize their weightings to determine if the image 
received is a match to some known object in its database. 

[0022] The technique of analyzing the reflectance function 
in terms of spherical harmonic components follows. 

[0023] Given L(n t ) , the intensity of light incident in the 
-n. direction on a perfectly reflecting, smooth surface with 
outward pointing normal n, the reflected light r (n e n) in 

direction n e can be expressed as 

r (n e ,n) = M(n e •n)|^.Z(n / )^(n e -n r ) 

where ii r = 2n(n,. n)-n, is the direction of specular reflection 
corresponding to incoming light along -n. . (Note Allies in 
the (n,n f .) plane, n r n r =1 , and nn^nn. Here u(x) is the 
unit step function: u(x) = 0,x < 0,u(x) = l 9 x > 0 . ) The pref actor 
w(n--n) ensures that the orientation n of the surface 
permits the reflection of light in direction n e . As 
(n e -n r ) = R(n^-n.)/ where = 2n(n e n)~n e and R is a rotation of 
n about n , S(n e -n r ) = S(n e -n f ), and since 

^(A 1 -A 2 )-S /m ^(A l )^I(A 2 X 



where Y im (n)is the 1m spherical harmonic evaluated in 
direction n, r (n e9 n) can be expressed compactly as 

where the L lm = J dn^n^Y^in .) are the components (strengths) of 
the incident intensity expressed in spherical harmonics. 
Choosing n e to be the z axis, and noting that n e =Rn e is now 
simply the rotation of the z axis about n by k, Y lm (Rz) 

becomes Y lm {fi,a\ where a and fi are the first two Euler 
angles of R. This is discussed in more detail below in 
connection with the Phong model. Note especially that as 
the polar angle of the surface normal n varies from 0 to 
71/ 2, the polar angle p varies from 0 to tc, as expected for 
perfect reflection. 

[0024] For comparison, one can derive the reflected light 
from a Lambertian surface r L (n e? n) in a similar manner. 

r L (n e ,n) = u(n e .n)J^(n,0(fi n, v 0) 

where (AvB) = max(A, B) . A Lambertian surface scatters the 
incident light equally in all directions n e such that 
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(n e -n)>0. The (n-n.) factor weights the incident flux 
density in the -n. direction. Since 

and (nn ; vO) = ^(4n/(21 + l)) U2 k l Y^(n i )Y lm (&) one can obtain the 
Basri- Jacobs (B- J) Expression: 

r t (n.,n) = U (K ■ A)£ fa I ta (4» 1(11 + 1)?' 2 *,7«. (*) 

The 1 coefficients calculated by B-J fall off as ' , and 
vanish for if ' is odd. This, coupled with constraints 

on the size of the i/m (as L(n.)>0) enabled them to truncate 
the sum over ' at ^-2. 

[0025] Such a truncation is clearly not possible for 
specular reflection; however, apart from mirrors, such 
reflection is seldom present. One model which retains 
much of the directionality of specular reflection, but 
introduces a controlled amount of lateral diffusion is the 
Phong Model, in terms of which one can write 

r P (n e ,n) = u(n e -A^daM*^'*, v 0)(n e ri, v 0/ 
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where as above n r = 2n(n ( . •&)-&. . Here the last term 

distributes the incident light into cones of decreasing 
intensity about the nominal direction of (specular) 
reflection n r : the larger the exponent p, the sharper the 
distribution. This expression reduces to r L in the limit 
that p->0. As such it may over-emphasize the diffuseness 
of the scattering. Also, Phong scattering is intended to 
augment rather than replace Lambertian scattering. An 
expression that reduces to in the limit p-^oo is: 

r p (n e ,n) = u(n e ^dn i L(n i )u(n^n i )(n e ri r v 0) p 

which may overemphasize the specularity of the scattering. 
Fortunately the analysis is valid for both: one need only 
substitute a kj 0) for k x to obtain r p from r p . 

[0026] In the above manner one can write 

(n • A, v 0) = (4* 1(21, + D) 1/2 *, ^ (A)^ (A, ) 
u(n • n, ) = Z lm V* + ») m n„ h (*, ) 
(n. • n, v 0)" = X km2 ^ + O)" 2 ^ > 

where k { is calculated in B-J Appendix A, and fc/ p) in B-J 
Appendix C, where a is used in place of p. £/ 0) can be 
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obtained analogously, and is given below, along with k, 
is given below as well. Combining one can obtain 



r p (n e ,n) = u(n e -n)^ im L lm 
(4*/(2/, +l)) , %^ (i (n) 
•X /2m2 (4^/2/ 2+ l)) 1/2 ^% l2 (A e ) 



where the remaining integral over incident direction (n 
independent of (n e ) . For the purposes of the above 



X X,2 I2 1 = 0 

(tt/3) U2 1 = 1 

*iH(-ir((2t + »rg-2)i even 

2'(//2-l)!(//2 + l)! 

0 / > 2, <«W 



.1/2 



jt< 0) = ■ 



(-\r + ' v \{2i+\)7ty"{i-\)\ 

2 1 ((/ - 1) / 2) ! ((/ + 1) / 2) ! 
0 



/ = 0 

/ > 0, odd 
I > 0, even 



and 
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0 



p<l,{l-p) 



even 



{{21 + \)7tf 2 {-\f + ?^ l2 p \{l- p -\)\ 
2 l {{l-p-\)l2)\{{l + p + \)l2)\ 



p<l,{l~p) 



odd 



{{ip+yxf^^pKp-iy. 

{2p + l){2p-l)l 



p = l 



{{2l + l)xf 2 2 M p\{{p+l)l2)\ 
{p-l\p + l + \)\{{p-l)l2-\)\ 



p>l,{p~l) 



even 



({2U\)7zf 2 p\{{p-l-\)l2)\ 
2 /+1 (p-/)!(( J p + / + l)/2)! 



p>h(P~l) 



odd 



[0027] Without loss of generality one can take the 
direction of observation -n e to be the -z axis of our 
coordinate system, as is commonly chosen. This is also the 
system in which the lighting is specified. As Y lm (z) is zero 
unless m = 0 r ^ 0 (njmust be expressed in terms of Y lm (n t ) in 
order to carry out the final integral over n f . 

[0028] Proceeding as follows: n r = 2n(n i n)-n, = Rn / represents 
a rotation R of n. around by n by n (or by -k) . Though it 
may intuitively seem more natural to view the relation 
between n, and n r as a rotation in the plane of incidence 
defined by n. and n about an axis through n ; xn, using R 
instead effects numerous simplifications. From the 
relation between n r and n. it is clear that 



R = 2nn T 



-I 
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whose representation in terms of Euler angles can be found 
by first equating gentries 

cos f3 = R 33 = 2n 2 z - 1, sin (3 = 2 \ n z | (1 - n\ ) I/2 

and then equating R 3l entries 

R n = 2n z n x = sin/?cosa; cosa = n z n x l\n z \{n 2 x + n 2 v ) U2 

[0029] Since R is symmetric, cosf =-cosa and sin^ = sina. 
Thus y=(x-a). Also, sirm = n z n v I \ n z | n 2 x + n 2 y ) V2 . (Here (a 9 /3 9 y) 
are the three Euler angles representing R, and n = (n x9 n y ,n z ) . 
Strictly speaking, the Euler angles refer to a rotation of 
the coordinates axes while the vectors remain fixed. 
Hence, one should be concerned with representing R r ; 
however, since R is symmetric, R = R r .) Since z*n>0 ? n z >0, 
and n z /\n z \=l . 

[0030] Now, to evaluate ^ 0 (Rn r ) , the polar angle 9 r between 
n f and the rotated z-axis is needed. However, in the 
original system, the rotated z-axis lies in the 

=(0==/?>0 = a) direction, and n, is unchanged. Hence 
cos^^n^n^ and, as can be shown 

Y l0 (n r ) = r w (Rn, ) = (4* /(2/ + if J.^ (fi t aWM 
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The expression for the reflected light based upon the Phong 
model then becomes 

/ >(^ A )= M ( £ - A )Zfa Zr te 

.X^(4^/(2/ 1+ l)) 1/2 ^^(n) 

It remains to evaluate the latter integral. 

[0031] The latter integral can be readily performed if the 
product of the second pair of spherical harmonics can be 
written as a sum of spherical harmonics. Such problems 
were solved in the context of the theory of angular 
momentum, a key component of present understanding of 
atomic spectroscopy. The result for this integral is that 
it is equal to 

((21, +l)(2/ 2 +l)/4^(2/ + l)) I/2 C(/ 1 / 2 /;m 1 m 2 m)C(/ 1 / 2 /;000) 

where the C are Clebsch-Gordon (C-G) coefficients; simple, 
real numbers, readily calculable, and equal to zero unless 
m = rn l +m 2 ,\m { \ < l x \m 2 \ < l 2 ,\m\ < I , and / equals one of 
(l { +l 2 ),(l x +/ 2 -l),...,^ -l 2 \ - This is known from previous work in 
angular momentum theory. 
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[0032] The next step is to consider, based on the values of 
k, and kj p) , which values of the OG coefficients will be 
necessary in evaluating the sums over (l v rn x ,l 2 ,m 2 ) . Again, 
from work in angular momentum theory, the following is an 
expression for the OG coefficients: 

C(lJ 2 l;m { m 2 m) = S m ^ m2 

f (/ + /, -/ 2 )!(/~/ 1 +/ 2 )!(/ 1 + l 2 -l)\(l + m)\(l-m)\ T 

\ l + ) (/ 1 +/ 2 +/-fl)!(/ 1 -m 1 )!(/ 1 +m 1 )!(/ 2 -^ 2 )!(/ 2 +m 2 )! y 

*^ ^ q\(l-l x + / 2 -q)\(l + m~q)\(q + l x -l 2 -m)! 

where q are the integers satisfying 

0v(/ 2 -l { +m)<q<(l + m)A(l 2 -l { +/) 

where A v B =max(A,B) and C aD =min(C,£>) . As usual, the 
convention is that 0!=1. Other useful facts about C-G 
coefficients are that C(l { 0l;m { 0m) = d hl S^ m , and that C^/jOOO) = 0 

unless (/ 1 +/ 2 +/) is even. 

[0033] Consider the 1 = 0, m = 0 factor (of L 00 ) in r P (z 9 n) , the 
d.c. response, so to speak. Since / = 0 3 / 1 =/ 2/ and as m = 0 9 
m 2 = -m } = m l one can either evaluate 

C^O^O) = ^^(-1)*^ /(2/, +1)' /2 
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or perform the above integration over incident-light 
directions 

to obtain for / = 0, m = 0 : 

Thus, unlike the cases of Lambertian and specular 
reflection, even for uniform lighting conditions, the 
reflected light is a function of the surface normal n . 
This is in addition to the effect of the nonlinearity due 
to the w(z-n) term, which enters in all cases. Do note, 
however, that k h decreases as , while ^changes 

comparatively slowly until l x > p 2 , where upon it falls 
roughly as exp-lf I p reaching T P by l x =p 9 and then falls off 
as 1/Zf for l x >p. Thus, the major n-dependent term in the 
(l = 0,/n = 0) -component of r p will be the ^=1 term in the above 
sum. (The sum over m { is, of course, real.) For the case 
of r p ^ ] falls off as III,. 

[0034] One might wonder how, if the lumination is uniform 
in solid angle (Z 7m =0,/>0) f one could possibly have an 
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angular dependence in the reflected light. However, any 
scattering which depends on, and only on, the angle to the 
surface normal will exhibit a non-uniform distribution when 
viewed from different directions. For example, if light 
were scattered only in the direction of the surface normal, 
regardless of incident angle, then a sphere would appear 
dark except for an intense bright emission from the center 
of the apparent disk. This is easily seen analytically. 
Choosing n e =z, the expressions for r s ,r L9 r P have the generic 
from 

r(n) = \ rfn ; i(n)g(n ; ,n) = £ lm L lm G lm (n) 

[0035] Thus G 00 = Jin,.g(n,.,n)/(4^) 1/2 and since g p depends on 
(n,.-n), (n c -n,), and (n e -n), the integral over n, , (4^) 1/2 G 00 , 
will be a function of (n n e ) = (n z) . Hence, although G 00 is 
independent of the azimuthal orientation of n, in most 
cases it will depend on n's polar orientation. Since g L 
depends only on (Vn), G o0 for the Lambertian model is 
independent of n; the tight coupling between the incident 
and reflected light in specular reflection yields 
G 00 =F 00 (n^) = (4^)" 1/2 a constant. However, for most g(n,,n), 
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G 00 =G 00 (n-z) . Without the (Vn) constraint, this directional 
dependence can be suppressed; however, scattering from very 
oblique surfaces can be enhanced as much as a factor of 2. 



[0036] See also the coefficients of L lm and L 2m in the 
expression for r P (z 9 n) . For 1=1, the only possible values of 
(Ii, 1 2 ) are l x = (1 2 +D and 1 2 = {li+D , (/,a/ 2 ) starting from 
zero: 2i = 1 2 is out as 1 ± + 1 2 + 1 = 21i -f 1, which, being 
odd, renders Ci^lil; 000) = 0} and li - 1 2 + n or 1 2 = li + 
n yields a smallest 1 of n, hence n > 2 are out for 1 =1 . 
Also, as 

C{l x l 2 l\m x m 2 m) = (-if^CilJJim^m) 

(exchange), one can easily obtain the li = (I2 + 1) case. 
Using Rose's expression, find: 

c(/ 1 (/ 1+ oi;iii 1 («-m 1 )« = (-i) ( (2/i+3)!(/i _ mi)!(/i+mi)!(1+ ^ )!(1 _ m)! ) 

in which, -l x <m x <l x and -1<w<1. Using this, one can obtain 
for the (I = 1, m) term in the Phong-model reflectance: 

,,(i3) 1 .=»(i-fi)A,(-ir<4-/3)-E,,^ T3 ^^ T ]) 

(1 + m + Z, -m^lCl-m + Z) t , 2 

(/,-/»!)!(/! +«,)!(! + w)!(l-w)! 
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In this (1 = 1) expression, the sum on 2i runs from l x = 0 , 
and the sum on mi runs from m± = -1± to m x = As with 

1 = 0, the terms in the 1 2 sum fall off as l/l* +p for 1 } > p . 

[0037] For 1 = 2, the Clebsch-Gordon coefficients are more 
complicated. We will simply state the result for this 
component of the reflectance. Note that only 1 2 = li + 2, 
11=12 + 2 , and li = I2 must be considered, and that the 
first two are, or course, closely related. The third case 
is more involved and requires separate expressions for m = 
+2, m = ±1, and m=0 . 



r p (z,n) 2m = u(z • n)L 2m (-If (4* /5) 1 ' 2 ^ / 



+ m + -m,)!(2-w + /i +^i)! y / 2 
(2 + m)!(2-m)!(/, + «,)!(/, -w,)! 



+ x>r 3 (2?V 3 ?! ! (((/i +2)))1 



1/2 

-(it/, -dm*, + du, -1- £1)1 

(2/, +3)! 

((/, + ™i )(/, + m, - 1)(/, - w, + 2)(/, - m, + 1)) ,/2 

<^*U + (A)} W 2 {f3 > a)) 

+m +i)) 1/2 E Wl (( / 1 +w .x / l +D) ,/2 (i-2«o 

+|a (A + (A (A + 1) - 3w i GO*?, » I/2 

[0038] In this (1=2) expression, the terms in the first ( 1 2 
= lx+2) , second (1 2 = li, m = 2) , and fourth (1 2 = li, m = 0) 
Ix-suiraaations all fall off as Ml?" , as for 1=0,1, while 
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those in the third (I 2 = l lf m = 1) fall off as III, p . The 
unprimed sum on li is over Ii = 0, and the primed sum is 
over li = 1. The unprimed sums over mi are from -l x <m x <l {f 
the single primed sum is from -l x +\<m x <l {f and the doubled 
prime sum is from —l l +2<m l <l ] . 

The expression 

C{l x l 2 l\m { m 2 m) = C^liin^m) 
was useful in the above derivation. 

[0039] From the foregoing, the function of broadened 
specular reflectance, r p (z,n) , can be described in terms of a 
summation of the spherical harmonic components of the light 
incident upon the model. 

[0040] It is useful to consider the spherical harmonic 
components of the reflected light as images of a given 
model, nine in the exemplary embodiment, each image 
produced by light reflected from the model when illuminated 
by only one spherical harmonic component of incident light. 
The complete set of images that model can produce can be 
approximated by some linear combination of those nine 
images. Therefore, matching a model of the object to some 
input image becomes a matter of optimizing the coefficients 
of each harmonic component image so that the summation of 
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the components creates a rendered image that most nearly 
approximates the input image. 

[0041] The method of deriving the reflectance function 
described above is used in the object recognition process. 
To match an input image with an object model in the 
database, the object model must be positioned to match the 
position of the input image. There are standard methods of 
performing this function known in the art, though a 
preferred method is disclosed in U.S. Patent Application 
Serial Number 09/538,204 (Attorney Docket No. NECI 1083, 
13414), filed 30 March 2000, entitled "Method for Matching 
a Two Dimensional Image to One of a Plurality of Three 
Dimensional Candidate Models in a Database", the disclosure 
of which is hereby incorporated by reference. 

[0042] Once positioned, images of the model are rendered 
under each of one or more spherical harmonic components of 
light as described above. These component rendered images 
can be linearly combined to arrive at an optimal rendered 
image of the model. An optimal rendered image means a 
rendered image that most closely approximates the input 
image. The comparison of the various rendered images to 
the input image is made by calculating the Euclidian 
Distance between the images, in a manner that is known in 
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the art. The optimization is performed by a least squares 
fit technique between the vector that is the reflectance 
function and the matrix that represents the input image, 
also known in the art. 

[0043] Other steps follow the optimization in the 
recognition process. One is computing some measure of 
similarity between the input image and the optimal rendered 
image for each model. This is accomplished be measuring 
the Euclidian Distance between each optimal image and the 
input image. The object model whose optimal rendered image 
is nearest the input image, assuming some threshold value 
of similarity must be reached, is a match to the input 
image . 

[0044] While the reflectance function of the present 
invention is described in application to the field of 
machine vision, it is not limited to that field. There are 
numerous parallels between this work and advances in the 
field of computer graphics, and our invention has 
applicability there as well. The field of computer 
graphics is continually striving to produce more realistic 
rendered images based on object models that exist only in 
the computer. The reflectance function of the present 
invention advances that work as well. 
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[0045] Our invention has been described herein with 
reference to a particular exemplary embodiment. Certain 
alterations and modifications may be apparent to those 
skilled in the art, without departing from the scope of the 
invention. The exemplary embodiment is not meant to be 
limiting on the scope of the invention, which is defined by 
the appended claims. 
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